Fix bug that header values are not null-terminated #970
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are currently two bugs in the producer code when dealing with buffers as header values.
The first issue is that the data of buffers passed as header values is cut off on the first null byte. According to the documentation (see https://kafka.apache.org/documentation/#recordheader) headers of Kafka messages can be arbitrary byte sequences. The bug is caused by the use of the copy constructor in
std::string
, which stops copying on the null byte. Using the copy constructor where the length of the byte sequence is passed explicitly will fix this issue.A second issue is that a buffer is treated similar to a string and that its content is passed to the UTF8String constructor. This will modify the byte sequence, as this constructor will transform the passed data from the node internal UTF-16 string representation into a UTF-8 representation. Since buffers can be arbitrarily encoded data and not necessarily strings, buffers should not be passed to the UTF8String constructor. Instead, their value should be used as is.